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fSJ 1 Abstract 

This paper studies the recovery guarantees of the models of minimizing + ^ 1 1 <-f 1 1 |r where ^ is a tensor 

and || A" ||* and ||<-f ||f are the trace and Frobenius norm of respectively. We show that they can efficiently recover 
low-rank tensors. In particular, they enjoy exact guarantees similar to those known for minimizing \\X\\* under the 
conditions on the sensing operator such as its null-space property, restricted isometry property, or spherical section 
property. To recover a low -rank tensor X°, minimizing ||#||* + ^jH-^Hf returns the same solution as minimizing 



almost whenever a > 10 max \\X^ || 2 . 
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I. Introduction 

Low-rank tensor recover problem is the generalization of sparse vector recovery and low-rank matrix recover 
to tensor data (H, El, 0. It has drawn lots of attention from researchers in different fields in the past several 
years. They have wide applications in data-mining, computer vision, signal/image processing, machine learning, 
etc.. The fundamental problem of low-rank tensor recovery is to find a tensor of (nearly) lowest rank from an 
underdetermined = b, where J is a linear operator and X 6 ^i^hx—xi N j s a _/\T-way tensor. 

To recover a low-rank tensor X° e R IlX 2X '" ^ from linear measurements b = a powerful approach is 

the convex model (2, |3| 

mmill*!!* :$(X) = b}, (1) 

A! 

where 

1 N 

W X W* '■= JfJ2\\ X (i)W* ( 2 ) 
i=l 
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and Xu\ is the mode-i unfolding of X, is the trace norm of the matrix Xu\, i.e. the sum of the singular 

values of X^y For vector b with noise or generated by an approximately low -rank tensor, a variant of dTb is lf3ll 

ndn{||Af|U:||ff(Af)-6|| 2 <(7}. (3) 

Pi 

Despite empirical success, the recovery guarantees of tensor recovery algorithms has not been fully elucidated. 
Recently, several authors H, (SI, 0] have got excellent results in the guarantees of sparse vector recovery and 
low-rank matrix recover. In this paper, we try to generalize these results to low-rank tensor recovery. To the best 
of our knowledge, this is the first paper that studies the guarantees of low-rank tensor recovery algorithm. 

This paper mainly studies the guarantees of minimization of the augmented objective \\X\\* + ^H^H^- The 
augmented model for (Q]) and (f3]) are 

wm{\\X\\* + -l-\\X\\ 2 F :$(X) = b}. (4) 
X 2a 

and 

min{||^||, + : \\$(X) - b\\ 2 < a}. (5) 

x la 

respectively. These are natural generalizations of the augmented model for vector and matrix data [4] to tensor 
case. 



II. Notations 

We adopt the nomenclature mainly used by Kolda and Bader on tensor decompositions and applications [711, and 
also a few symbols of (H, (3. 

The order N of a tensor is the number of dimensions, also known as ways or modes. Matrices (tensor of order 
two) are denoted by upper case letters, e.g. X, and lower case letters for the elements, e.g. Xij. Higher-order tensors 
(order three or higher) are denoted by Euler script letters, e.g. X, and element , in) of a iV-order tensor 

X is denoted by Xi t i 2 ...i N . Fibers are the higher-order analogue of matrix rows and columns. A fiber is defined by 
fixing every index but one. The mode-n fibers are all vectors x^...^ _ 1 -i n+1 -i N that obtained by fixing the values 
of {i\,i2,--- j*iv} \ in- The mode-n unfolding, also knows as matricization, of a tensor X G ^ixhx— xIn j s 
denoted by Xt n ) and arranges the model-n fibers to be the columns of the resulting matrix. The unfolding operator 
is denoted as unfold(-). The opposite operation is refold(-), denotes the refolding of the matrix into a tensor. The 
tensor element • • • , ijy) is mapped to the matrix element (i n ,j), where 

N 

j = 1 + ^2(ik ~ 1) Jk with J k = Y[ Im 

fc=l m=l 
fc^n m^n 
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Therefore, X( n ) G R 7 " xJl '" Jjr. xhe n-rank of a iV-dimensional tensor Af, denoted as rank„(,Y) is the 
column rank of X( n \, i.e. the dimension of the vector space spanned by the mode-ra fibers. We say a tensor X is 
rank (ri, ...,r/v) when = rankfc(Af) for fe = 1, ...,N, and denoted as rank(Af). We introduce an ordering among 
tensors by the rank:rank(A') < rank(3^) 4=> (ranki(Af), ...,rankjv(<^)) ^ (ranki (y) ranker (y) ) 44> rankj(A') < 
rankj(3^),i = 1, N. The inner product of two same-size tensors X,y G M 7 i x/ 2x - x/ N j s d e fl ne( j as vec(A') T • 
vec(^), where vec is a vectorization. The corresponding norm is ||^||f = \/< X , X >, which is often called the 
Frobenius norm. 

The ra-th mode product of a tensor X G ^hxi 3 x-xi N with a matr i x JJ ER JxIn is denoted by X x n U and is 
of size I\ x ... x I n _i x J x I n+1 x • • • x ijy. Elementwise, we have 

In 

(X X n U)i 1 ...i n l ji n+1 ...i N = 'y ^ x iii2—i N Uji n . (6) 

i n =l 

Every tensor can be written as the product (8j 

^ = 5x 1 C/( 1 ) x 2 U {2 \.. x N U^ N \ (7) 

in which: 

is a unitary I„ x J n matrix, 

• S is a Ji X 1% X • • • X iAT-tensor of which the subtensors Si n=a , obtained by fixing the n-th index to a, have 
the properties of: 

1) all-orthogonality: two subtensors Si n=a and Si n= p are orthogonal for all possible values of n, a and (3 
subject to a / j3: < Si n=a ,Si n=l 3 >= 0, 

2) ordering: for all possible values of n, one has ||«Sj ra =i||.F > ||5j„=2||F > ••• > ||«->i„=J„ ||f > 0. 

The Frobenius norms ||<Si„=i||fj symbolized by o{ , are mode-n singular values of A, that means the singular 
values of A( n y 

This is called the higher-order singular value decomposition (HOSVD) of a tensor A in (H. Some properties of 
this HOSVD which will be used in this paper are list below as lemmas: 

Lemma 1: (0 Property 6). Let the HOSVD of A be given as in (O, and let r n be equal to the highest index 
for which ||<Sj n=r J|.F > 0; then one has rank(^4( n )) = r n . 

Lemma 2: ( [8] Property 8). Let the HOSVD of A be given as in (0; due to the unitarily invariant of the 
Frobenius norm, one has ||.A||fi = Yl f "!^) = ••• = 2~2 f "!^) = II^IIf- 
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— Model (1) 
— <J.1||X°|| F 
<i=10||X || F 
Ct_2S||X°|| F 









(a) Recovery relative errors 
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(c) Image recovered via ([TJ 
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(d) Image recovered via © with q = (e) Image recovered via l(4j with a = (f) Image recovered via @ with a 
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Fig. 1. Facade in-painting. 
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III. Motivations and contributions 
To explain why model (@]) is interesting, we conducted following tensor completion simulations 



minillAflL + — : x 



x 



2a 



F • X «l22...lN 



m il i 2 ...i N ,hi2---iN e 



(8) 



to compare it with model (Q]) based tensor completion 



min{||^||* : x ili2 ... iN = m il i 2 .. AN ,i 1 i 2 ...iN £ 



(9) 



The facade image data of (3l was used here to be an example. Models ([8]) and (O were solved to high accuracy 
by the solver LRTC (3). For each model, we measured and recorded 



recovery relative error : \\X* — Xo\\f/\\Xq\\f- 



(10) 



The relative errors are depicted as functions of the number of iterations in Figure 1(a). 

Motivated by the above example, we show in this paper that any a > 10 max ||^( j) lb guarantees that problem ((4]) 
either recovers X° exactly or returns an approximate of it nearly as good as the solution of problem (OQ). Specifically, 
we show that several properties of 5 r , such as the null-space property (a simple condition used in, e.g., 0, ifTOl . 
ifTTll . Ifl2l . lfT3l ). the restricted isometry principle lfl4l . and the spherical section property lfl"5l . which have been 



used in the recovery guarantees for vectors and matrices, can also guarantee the tensor recovery by model £[)) 
and ©. 

Even though X° not known when a is set, max H2 is often easy to estimate. When max||X^||2 is not 
available, using inequalities ||A' ||i? > max||X^||2, one get the more conservative formulae a > 10||Af°||.F. 
Furthermore, when $ satisfies the RIP, one has < /3||6||2 for some f3 > 0; hence, one has the option to use 

the even more conservative formula a > 1 /5 1 1 1 1 2 - 

IV. Tensor recovery guarantees 

This section establishes recovery guarantees for the original and augmented trace norm models CO) and (@]). The 
results are given based on the properties of $ including the null-space property (NSP) in Theorem [5] and [6j the 
restricted isometry principle (RIP) lfl4l in Theorem [8] and |9j the spherical section property (SSP) lfl31 in Theorem [TOl 
These results adapt and generalize of the work in |4). 

A. Null space property 

The wide use of NSP for recovering sparse vector and low-rank matrices can be found in e.g. iflOl . ifTTTl . |[T2l . 
|[T3l . In this subsection, we extend the NSP conditions on $ for tensor recovery. Throughout this subsection, 
we let o~i(X),i = l,...,m denote the i-th largest singular value of matrix X of rank m or less, and SpT) = 
diag(<Ti(X), ...,a m (X)) denote the diagonal matrix of singular values and s(X) = (ai(X), ...,a m (X)). \\X\\2 = 
<Ji(X) denotes the spectral norms of X. 

We will need the following two technical lemmas for the introduction of the tensor NSP conditions. 

Lemma 3: ([8] Theorem 7.4.51). Let X and Y be two matrices of the same size. Then we have 

m 

J2\^i(X)-ai(Y)\<\\X-Y\U (11) 

i=i 

and 

m 

X>i(X) - ai(Y)) 2 < \\X - Y\\ 2 F . (12) 
i=i 

Lemma 4: ([4] Equation (19)). Let x and h be two vectors of the same size, S := supp(x) and Z = S c . Then 
we have 

\\x + h\\i > \\x\\i + \\hzh - \\hs\\i (13) 



and 

1 .. . ... r .. 1 „ .,i 

- + 



\ x + Hi + 7^-\\x + Hi > 
la 



Mil + 7T X II2 
2a 



IMi-(i + ™^)ll^lh 

a 



+ ^\\h\\l (14) 



Now we give a NSP type sufficient condition for problem fl}. 



Theorem 5: (Tensor NSP condition for £[])). Assume X° is fixed, problem (Q]) uniquely recovers all tensors X G 
of rank (n, •■•,rjv) or less from the measurements 5 r (A' ) = 6, if all % G Null(5 r )\{0} satisfy 

N li N n 

E E ff j( 5 w)>EE#«)- (15) 

j=l j=r'i+l i=l J=l 

Proof: Pick any tensor <Y° of rank (ri, rjy) or l ess an d let 6 = 5 r (A'°). For any % G Null(5 r )\{0}, we have 
${X° +%) = = b. By using CCD, we have 

N N 

n*° + ^ii* =^ E 114) + H ®\\* > jr E ik4) - s (^«)iii 

i=l i=l 

1 N li Ti 

^£(ii4)H* + [ E «-E#w)i) (i6) 

i=l J=r,+1 j=l 

=ii^°iu + ^E( E «-E#(o)) 

i=l j=r s +l j=l 

where the first inequality follows from ([T3T ). For any nonzero G Null(S), ||^|||, > 0. Hence from {B} and flg), 
it follows that X° is unique minimizer of (fl). ■ 
We can extend this result to problem (H} as follows. 

Theorem 6: (Tensor NSP condition for (0])). Assume X° is fixed, problem (@]) uniquely recovers all tensors X° 
of rank (n, ...,r/v) or less from the measurements 3^°) = 6, if all % G Nun(30\{O} satisfy 

N h N llj^O II n 

E E «^E(i+^)E«- ( 17 ) 

i=l j'=r>+l i=l j'=l 

Proof: Pick any tensor Af° of rank (n, ...,rjv) or less and let 6 = 5 r (^°). For any nonzero % G Nuil(30, we 
have $(X° +H) = = b. Thus 

1 1 N 1 

||*° + H\U + -||*° + = ^ E(H4) + ff (0ll* + 2^114 + h ®Wf) 

1=1 

1 - 1 

E(ii s (4)) - s (^))iii + ^ii s (4)) - s ( ff «)iii) 

i=l 



(18) 



^^E([ii4ii* + ^H4)ii^ + [ E w - a + — ^-) + ^ii^nl) 

i=i j=n+i j=l 

=ii^iu + ^p iii + ^ii^iii + ^E( E ^(^))-d+^)E-i(%)) 

i=l j'=ri + l j=l 

where the first inequality follows from (TTTb and (fT2l) . and the second inequality follows from ( fT"3T ) and ([T4l ). For 
any nonzero U G Null(J), \\H\\ 2 F > 0. Hence, from (HU) and ([17]), it follows that X° + H leads to a strictly worse 
objective than X°. That is, X° is the unique solution to problem (01). 
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Remark 1: For any finite a > 0, (fTTT ) is stronger than (fl~5T ) due to the extra term — Since various uniform 
recovery results establish conditions that guarantee (fT5b . one can tighten these conditions so that they guarantee (fTTl) 
and thus the uniform recovery by problem (|4]). How much tighter these conditions have to be depends on the value 



B. Tensor restricted isometry principle 

In this subsection, we generalize the RIP-based guarantees to tensor case and show that any a > 10 max ||^( j) H2 
guarantees exact recovery by (01). 

Definition 1: (Tensor RIP). Let £PT( ri , r3 ,..., rjv ) := {X € K /iX/2X, " x/jv : rank(X (n) ) <r n ,n = 1, N}. The RIP 
constant 5( ri ,r 2 ,...,r N ) °f linear operator J is the smallest value such that 

))\\xf F < \\d(x)§ <(i + s ir 

i,r 2 ,...,r N ))\\X\\f (19) 

holds for all X G £Dt( r r rjv \. 

The following recovery theorems will characterize the power of the tensor restricted isometry constants. The first 
theorem generalizes Lemma 1.3 in |[T4l and Theorem 3.2 in[5] to low-rank tensor recovery. 

Theorem 7: Suppose S^ ri> 2r 2 ,...,2r N ) < 1 f° r some (n, r-i, t/v) ti (1, 1, 1). Then X° is the only tensor of 
rank at most (ri,r2, ...,rjf) satisfying $(X) = b. 

Proof: Assume, on the contrary, that there exists a tensor with rank (ri,r2, ...,rjv) or less satisfying $(X) = b 
and X 7^ Afo- Then Z := X — Xq is a nonzero tensor of rank at most (2ri, 2r%^ 2rjy), and ^(-Z) = 0. But then 
we would have = > (1 — S(2ri,2r 2 ,...,2r N ))\\^' \\f ^ which is a contradiction. ■ 

The proof of the preceding theorem is identical to the argument given by Candes and Tao and is an immediate 
consequence of our definition of the constant <5( riir2 rjv ). No adjustment is necessary in the transition from sparse 
vectors and low-rank matrices to low-rank tensors. The key property used is the sub-additivity of the rank. Adapting 
results in 131, @, we give the uniform recovery conditions for (HJ below. 

Theorem 8: (RIP condition for exact recovery by (Q])). Let X° be a tensor with rank (n, r2, rpf) or less. 
Problem (Q]) exactly recovers X° from measurements b = $(X°) if 5 satisfies the RIP with S^ Il ^r n ,. ..,i N ) < 0.4931, 
for n = 1, ...N. 

Proof: For any nonzero U £ Null(ff), \\H\\f > 0, let U = Sx 1 U^ x 2 U ( - 2 \.. x N IfW be the HOSVD of U. 
From Proposition 3.7 of lfT6l we have rank(i?( n )) < rank(5( n )), (n = 1, .., A r ), thus we have rank('H) X rank(5). 
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We decompose % = Hq +%\ + where 

U = S {in a,...,r n } xi U w x 2 U (2 K.. x N 17 W, 

(20) 

Hi = 5 {i „ :rn+1) ... )2r „ } xx *7« x 2 C/( 2 )... xjv C/ (JV) , 

and Su. a \ is the tensor obtained by fixing the n-th index to the index set a, others to zero. Similarly we have 
rank(%)i ^ rank(<S/j n .^ rw _|_i )###) (j +1 y„}) ^ ...,r n , I n ), (i = 0,1,...). Due to the unitarily invariant of the 
Frobenius norm, we have ||%o||f = H<%„ : i,...,r„}llF, \[Hi\\f = \\S{i n -.r n +i,...,2r n }\\F, .... Let /i = {af\a^\ ...,afj) 
and /i = (<rp,...,<r^), ^ = (o"^ x , — ,cr 2 ™„), /i 2 = (o" 2 r„+i' — ' a 3r„)>—> wnere °i is the i-th largest mode-n 
singular value. From the definition of HOSVD and Lemma 2, we have 

(i+l)r„ 2 

11^111 = ll%„ : ^+i > ...,( i +i)r„}lll= E =\\hi\\i(i = 0,l,...). (21) 

Due to the mean-inequation, one has ||/ij|| 2 > ||/ij||f/r ra , (i = 0,1,...). Assume that \\hi\\i = t(J2i>i ll^illl) with 
some t € [0, 1]. Then we have (£ l>2 \\hi\\i) = (1 - t)(E f >i INI0- 



First from Lemma 2.1, Lemma 2.2 in |6] and (1201 ) we have 

Ei%&< E Mi < f (i - *)(E imio 2 ( 22 ) 

i>2 j>2 n i>l 

and 

rkUi\\F < \\h l \\ 1 +r n {a ( £l +1 -a^l +i+rn+i )/A,i = 2,3,.... (23) 

From d23l we have 

rlEii^n^ < Eii /ii ii i+r "i^+ i i/ 4 ^ Eii^ii i + n /ii ii i / 4 = ^-^En^ii 1 - (24) 

i>2 i>2 i>2 i>l 

So we have 

||ff(7£ + > (1 - <y ( /xA,...,2r B ,...,/ w ))ll«0 + Hllll 

= (l-%,/ 2 ,.., 2 r„,..,/ N ))(ll^o||| + ||^i|||) (25) 

> (l-%,/ 2 ,..,2r„,...,/ w ))(ll^0||l + ||^l||i)An 
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and 

\\dm 2 + n 3 + ...))\\ 2 2 = E <3{n j )^{n k )> 

j,k>2 

=j]<^),^)>+2 x: <^),5 r (H fe )> 

i>2 2<j<fc 

< E( 1+ %^--^--^))ii^ii^+% 1 ^ I -^,.. 1 /,) E 11^11^11^11^ 

j>2 2<j<k 

i>2 \i> 2 



(26) 



Further more, by (I22l) . (|24t we have 

||^ 2 +^3 + < E WKiWr + ( E ll^i^ 

i>2 \j>2 

<^(! " *)(E H^lli) 8 + (1 - E IMIi (27) 

i>l i>l 
*(1 ~ t) + *■/./, 2, ~ 3t/4) 2 

= ; (Ell^ll 1 ) • 

r ™ »>i 

Since 5(^o + Ui + U 2 -) = 3(H) = 0, we have \\$(H + Wi))||| = 11^2 +% + .»)) 111. b Y the above equations 
we have 

(l-^ 1 ,/ 2 ,...,2,„,...,/ J v))(ll^0||? + ||/ i l||?) An 

. t(l-*) + ^ 1 ,/ a ,-,2r„-,/ w )(l-3t/4)% v . ... 2 (28) 

< (2^ HMO • 

r ™ i>i 

Hence, let 5 = 5(/ 1 ,/ 2) ... )2 r„,...,/ w ) 

H^Hl ^ 7^ [* + (!" 35 /2)t - (2 - 255/16)i 2 ] ||/h||i) 2 (29) 

i>l 

We have a quadratic polynomial of t with t G [0, 1] in the right-hand side of the above inequality. Hence, by 
calculus, this quadratic polynomial achieves its maximal value at t = ^z^jg £ [0) !]• Therefore we obtain 



HMi INK), where 

i>l 



y (1 — 5)(32 — 25<5) " 
5 < (77 - \/l337)/82 « 0.4931, then 9 < 1, we get ||/t ||i < (E INIl). whicn is 



4(1 



i>l 



E *r>Y,°i- < 31 > 

j=r„ + l jf=l 
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If for all n = (1, N), we have (fJT]), then we get ([151). ■ 
Next we carry out a similar study for the augmented model (01). 

Theorem 9: (RIP condition for exact recovery by (01)). Let X° be a tensor with rank (n, r 2 , rjv) or less. 
The augmented model © exactly recovers A' from measurements b = $(X°) if 5 satisfies the RIP with 
\h,...,2r n ,...,i N ) < 0.4404, n = l,...N and a > lOmax || 2 . 

Proof: The proof of Theorem [8] establishes that any nonzero H G Null^) satisfies ||/io||i < Q(Y1 II ^* Hi)- 

i>l 

Hence, if (1 + !£«^)6I < 1, notice < 1, we have 



, X° 2 ^4 1 + 5,5-452 

a > -i X° 2 = — (t) = (32) 

W ^(1 - <J)(32 - 255) - ^4(1 + 55- 45 2 ) 



For 5 = 0.0.4404, we obtain (0~ l - 1) _1 ||X .|| 2 « 9.9849||X ( ° ) || 2 < a, which proves the theorem. ■ 
Remark 2: Different values of 5(/ lt ... t 2r„,...,Jjr)> n = 1; ^ are associated with different conditions on a. Follow- 
ing (l32l . if ^/j,...^^,..../^ < 0.4715, n = 1, ...A r , a > 10 max ||X^|| 2 guarantees exact recovery. If 5(/ l5 ... ! 2r„,...,Jj V ) < 
0.1273, n = 1, ..JV, a > 10 max || 2 guarantees exact recovery. In general, smaller 5(i 1 ,...,2r n ,...,I N )i n = lj —- ^ 
allows a smaller a. 



C. Spherical section property 

Next, we we derive exact conditions based on the spherical section property (SSP) (H, [fl~5l. There is not much 
discussion on spherical section property (SSP) for low-rank tensor recovery in the literature, here we present a 
SSP-based result. 

Theorem 10: (SSP condition for exact recovery by ©). Let # : M. IixI * x '" xIn — > R m be a linear operator. 
Suppose there exists A > such that all nonzero nonzero H G Null(30 satisfy 

m *>M (33) 



||^||f " v a 

Assume that ||X^|| 2 , (i = 1, ...,iV) and a > are fixed. If 

m >( 2 + — W_ ) 2 r i A J (i = l,...,iV) > (34) 
a 

then the null-space condition holds for all nonzero % G Null^). Hence is sufficient for problem to recover any 
X° with rank (ri,r 2 , ...,rjv) or less from measurements 6 = ^(Af ). 
Proof: Condition (flTT ) is equivalent to 

N It N 11^0 II n 

E E *i W ^ E( 2 + -f-) E os) 

i=l j=l i=l j=l 
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Since Yl Vj{ H (i)) < Vn\ Jl ^'(-^(i)) 2 ^ V^II-^M Wf* <E5]) holds provide that 

j=i V i=i 



JV I( TV lljfO II 

^ £(2 + — ^V^Pfylb- (36) 



a 

i=l j'=l i=l 

Now from (f33t and d34l ), one has 

llj^O II 

>max(2 + «_)07||?{|| F> (37) 

i a 

which is equivalent to 

1 N \\X° II 1 N \\X° || 

- K) II* > max(2 + - Ja-^lWlk > - £(2 + — ^L^H^ \\ F . (38) 

i=l i=l 

Thus the null-space condition holds. ■ 

V. Conclusion 

In this work we focussed on the recovery guarantees of tensor recovery via convex optimization. We presented 
general results stating that the extension of some sufficient conditions for the recovery of low-rank matrices using 
nuclear norm minimization are also sufficient for the recovery of low-rank tensors using tensor nuclear norm 
minimization. We extended the null-space property, the restricted isometry principle, and the spherical section 

property conditions to the augmented tensor recovery problems, and find that any a > 10 max IIXP.JU guarantees 

t W 

that problem (0]) either recovers X° exactly or returns an approximate of it nearly as good as the solution of 
problem (HI). 

There are some directions that the current study can be extended. In this paper, we have focused on the recovery 
guarantees of the exact case; it would be meaningful to also analyze the guarantees for the stable recovery. Second, 
generalize the linearized Bregman algorithm [17] for augmented sparse vector and low -rank matrix recovery JH to 
low-rank tensor case. Moreover, from our results, there is a big "gap" between the recovery conditions for matrices 
and tensors, and we need to fill this "gap" in future work. In a broader context, we believe that the current paper 
could serve as a basis for examining the augmented trace norm models in tensor recovery. 
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